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APPARATUS AND METHOD FOR INCORPORATING TACTILE CONTROL AND 
TACTILE FEEDBACK INTO A HUMAN-MACHINE INTERFACE 



1 BACKGROUND OF THE INVENTION 

2 [0001] The invention relates generally to human-machine interface devices. More 

3 specifically, the invention relates to an enhanced track pad that allows a user to manipulate the 

4 graphical elements of a graphical user interface (GUI) and provides tactile feedback to the user. 

5 [0002] Current computer systems allow graphical elements such as windows, icons, and 

6 menus to be manipulated through the use of a pointer controlled by a mouse. This interface 

7 allows an operator to indicate a single point on the display and to click (or double-click, etc.) one 

8 or more buttons on the mouse. The only feedback that the user receives about the success of an 

9 intended operation is through the visual display provided by the computer screen. 

10 [0003] Like a conventional track pad, a mouse-controlled pointer provides only an 

1 1 indirect way of manipulating the elements of a GUI. Users cannot move a graphical icon or 

12 window directly with their hands, the way they would move a piece of paper sitting on a real 

13 desktop. Rather, users must operate on elements of a GUI through the intermediary action of a 

14 mouse, which only allows users to point and click. Furthermore, a conventional mouse provides 

15 no tactile feedback about the operations being performed on the GUI. The only feedback that a 
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1 user receives about the setup and execution of an intended operation is through the video display. 

2 Thus, information transfer between a computer and a human operator is less than optimal. 

3 [0004] A touch screen video display would seem to remedy some of these deficiencies. 

4 A touch screen display allows a computer operator to directly "reach out and touch" the item to 

5 be manipulated, without any intermediary mechanisms. However, touch screens have their own 

6 deficiencies, which limit the rate of information transfer between the computer and human 

7 operator. Graphical elements have to be of roughly the same size as a human fingertip, which is 

8 wasteful of screen real estate, and fingers tend to obscure the graphical elements to be 

9 manipulated. Fingerprints, smudges, and other marks on the screen provide a constant 

10 distraction. Also, current touch screen video displays, like conventional track pads and mouse- 

1 1 controlled pointers, provide no tactile feedback of any kind to users. 

12 [0005] Thus, there is a need for a device that incorporates control and tactile feedback, 

13 which allows a more optimal interface between the user and the machine. 

14 SUMMARY OF THE INVENTION 

1 5 [0006] The invention relates to an enhanced track pad that allows a user to manipulate 

16 the graphical elements of a graphical user interface and provides tactile feedback to the user. 

17 [0007] The enhanced track pad comprises a substrate and a plurality of tactile elements. 

18 Each tactile element of the tactile user interface device comprises a means for sensing pressure 

19 from a user's finger to determine if the user has depressed the tactile element and a means for 

20 conveying tactile feedback information to the user. 

21 [0008] A user views a video display and provides inputs or commands to a computer or 

22 processor through the enhanced track pad. The user uses the enhanced track pad to manipulate 
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1 elements of the graphical user interface in a similar fashion as one would use a conventional 

2 track pad to operate on the video display's graphical elements. Additionally, the enhanced track 

3 pad provides the user with enhanced sensory information about the GUI elements on the display, 

4 including the relationship between the user's hand and the GUI elements. In operation, the user 

5 rests his hand on the enhanced track pad, and a representation of the user's hand appears on the 



6 video display. Tactile feedback information, such as elevations, vibrations, wave-like 

7 undulations, textures, and temperatures, which correspond to the graphical elements on the video 

8 display, are felt by the user's finger on the enhanced track pad. Using tactile feedback 

9 information, the user then performs operations such as selecting GUI elements by "tapping" on 
1 0 the enhanced track pad. 



1 1 BRIEF DESCRIPTION OF THE DRAWINGS 

12 [0009] For a more complete understanding of the enhanced track pad, reference is now 

13 made to the following detailed description of the embodiments as illustrated in the 

14 accompanying drawings, wherein: 

15 (a) FIG. 1 illustrates the operation of the enhanced track pad. 

16 (b) FIG. 2 is a schematic drawing of the enhanced track pad. 

17 (c) FIGS. 3 A and 3B illustrate the correspondence between tactile elements on the 

18 enhanced track pad and GUI elements on a video display. 

19 (d) FIG. 4 is an embodiment of the enhanced track pad using an array of pins. 

20 (e) FIG. 5 shows another embodiment of the enhanced track pad with a flexible 

2 1 substrate disposed over an array of pins. 

22 (f) FIG. 6 shows yet another embodiment of the enhanced track pad. 
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1 DESCRIPTION OF THE EMBODIMENTS 

2 [0010] FIG. 1 shows a schematic representation of the enhanced track pad 10 in 

3 operation. A user 90 views a video display 50 and provides inputs or commands 16 to the 

4 computer or processor 40 through the enhanced track pad 10. The user 90 uses the enhanced 

5 track pad 10 to manipulate elements 53, 56, 58 of the graphical user interface (GUI) in a similar 

6 fashion as one would use a conventional track pad to operate on the video display's graphical 

7 elements. Additionally, the enhanced track pad 10 provides the user 90 with enhanced sensory 

8 information about the GUI elements 53, 56, 58 on the display 50, including the relationship 

9 between the user's hand and the GUI elements 53, 56, 58. In operation, the user 90 rests his 

10 hand on the enhanced track pad 10, and a representation 95 of the user's hand appears on the 

1 1 video display 50. The representation 95 of the user's hand provides additional visual feedback to 

12 the user 90 and need not be to scale. Tactile feedback information 14, such as elevations, 

13 vibrations, wave-like undulations, textures, and temperatures, which correspond to the graphical 

14 elements 53, 56, 58 on the video display 50, are felt by the user's finger on the enhanced track 

15 pad 10. Using tactile feedback information 14, the user 90 then performs operations such as 

16 selecting GUI elements 53, 56, 58 by "tapping" on the enhanced track pad 10. 

17 [0011] As shown in the schematic drawing of FIG. 2, the enhanced track pad 10 includes 

1 8 a fixed substrate 20 that supports an array of tactile elements or taxels 30. Substrate 20 may also 

19 contain at least a portion of the control circuitry 22 and any other associated circuitry required. 

20 Each taxel 30 in the array receives input from a user and provides tactile feedback information to 

21 the user. To receive inputs from a user, each taxel 30 includes a means 32 for sensing pressure 

22 from a user's finger, to include devices such as piezoelectric devices and mechanical switches. 

23 To provide tactile feedback information, each taxel 30 includes a means 34 for conveying tactile 
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1 feedback information to the user, to include devices such as mechanical pins and 

2 microelectromechanical (MEM) devices. Depending on applications, these devices need not be 

3 particularly small as the size of the enhanced track pad is not necessarily a constraint. 

4 [0012] FIG. 3A and FIG. 3B illustrate how an array of taxels 30 disposed on the surface 

5 of the enhanced track pad 10 may correspond to GUI elements, such as a window 52, menu 56, 

6 and icon 58, on a video display 50. Each taxel 30 corresponds to a pixel, a fraction of a pixel, or 

7 a group of pixels on the associated video display 50. For example, each taxel 30 in the array that 

8 corresponds to the boundary of the window 52, consisting of edges 53 and a title bar 55, is raised 

9 to a particular elevation distinguish it from the windows contents and its background, which are 

10 of different elevations from the boundary. Through these elevated contours, the user is then 

1 1 provided with information that reinforces and is in consonance with that presented on the video 

12 display 50. The representations of the GUI elements 52, 56, 58 on a video display as described 

13 herein are used as an example only, and it is therefore to be understood that other suitable 

1 4 representations of GUI elements may be used. 

1 5 [0013] In one embodiment shown in FIG. 4, each taxel of the enhanced track pad 10 

16 includes a vertical pin 60 disposed in a sleeve 61. Planar substrate 20 supports the array of 

17 vertical pins 60. In this embodiment, information is conveyed to the user by varying the 

18 elevations of the vertical pins 60. Each pin 60 is elevated to a selected pin position p n to 

19 represent different features, such as shapes and colors, of the GUI elements on the video display. 

20 The elevation of each pin 60 is selected from a plurality of pin positions po, pi, . . pn, through 

21 the use of devices, such as piezoelectric devices or electromagnets. In another implementation of 

22 the enhanced track pad 10, tactile feedback information may be conveyed to the user through 

23 wave-like undulations or vibrations. The user performs operations such as selecting a GUI 



5 



Navy Case No. 84037 

1 element on a video display by depressing the vertical pins 60 of the enhanced track pad 10 

2 corresponding to the GUI element. To determine whether the user has depressed a pin, a 

3 pressure threshold is set for the vertical pins 60. When the pressure exerted on the pins 60 is less 

4 than the set pressure threshold, no inputs from the user are sent through the enhanced track pad 

5 10. When the pressure exerted on the pins 60 exceeds the set pressure threshold, the pressure 

6 from the user's finger is interpreted as an input or a command. For example, the user will be 

7 able to locate and feel the title bar of a window by using pressure less than the pressure 

8 threshold, but will move the window from one area of the video display to another by using 

9 pressure more than the set pressure threshold. 

10 [0014] FIG. 5 shows another embodiment of the enhanced track pad 10. In this 

1 1 embodiment, the enhanced track pad comprises a fixed planar substrate 20 that supports an array 

12 of vertical pins 60, a flexible substrate 80 disposed on the array of vertical pins 60, and the array 

13 of MEM devices are disposed on the flexible membrane. An advantage of this embodiment is 

14 protection of the array of pins from environmental concerns such as dust, dirt, and moisture. 

15 [0015] In yet another embodiment shown in FIG. 6, each taxel 30 of the enhanced track 

16 pad 10 includes two devices, a device for sensing pressure to determine if the user has exerted 

17 pressure on taxel 30 and a device for providing tactile information to the user. A piezoelectric 

1 8 sensor 66 measures the pressure exerted on taxel 30 by the user's hand. When the piezoelectric 

19 sensor 66 measures pressure greater than a predetermined pressure threshold, the pressure from 

20 the user's finger is interpreted as an input or a command. 

21 [0016] Still referring to FIG. 6, to provide tactile information, each taxel 30 comprises 

22 one or more microelectromechanical (MEM) devices 68, such as for example, actuators. Each 

23 MEM device 68 has at least two mechanical states, state V and state 'b\ Different textures are 
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1 constructed by placing the MEM devices 68 on each taxel 30 of the enhanced track pad 10 into 

2 different mechanical states. For example, by placing all of the MEM devices 68 in state 'a\ a 

3 particular texture is felt by the user's fingertip moving across the enhanced track pad 10. Placing 

4 all of the MEM devices 68 in state c b J produces a different texture. Other textures are 

5 constructed by placing different subsets of the MEM devices 68 on each taxel 30 of the enhanced 

6 track pad 10 in different configurations. For example, a static checkerboard pattern is 

7 constructed by placing a selected number of MEM devices in state V alternating with a selected 

8 number of MEM devices in state 'b\ The representation of textures as described herein is used 

9 as an example only, and it is therefore to be understood that other suitable representations of 

1 0 textures may be used. 

1 1 [0017] In addition, the MEM devices 68 disposed on each taxel 30 of the enhanced track 

12 pad 10 are not limited to switching between static configurations, as described above, to convey 

13 tactile information to the user. For example, an array of micro-mechanical cantilevers vibrating 

14 in unison at one of a plurality of frequencies may be used to convey sensory information to the 

15 user, while an array of cantilevers vibrating in a wavelike manner may be used to convey another 

1 6 sensory information. 

17 [0018] In all of the embodiments described herein, information is conveyed to a human 

18 user about graphical items on a video display. However, the source of information is not limited 

19 to computers but could be any machine. 

20 [0019] Clearly, many modifications and variations of the enhanced track pad are possible 

2 1 in light of the above teachings. It is therefore to be understood that within the scope of the 

22 appended claims the enhanced track pad may be practiced otherwise than as specifically 

23 described. 
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